DevOps in Practice -- A preliminary Analysis of two Multinational Companies

DevOps is a cultural movement that aims the collaboration of all the stakeholders involved in the development, deployment and operation of soft-ware to deliver a quality product or service in the shortest possible time. DevOps is relatively recent, and companies have developed their DevOps prac-tices largely from scratch. Our research aims to conduct an analysis on practic-ing DevOps in +20 software-intensive companies to provide patterns of DevOps practices and identify their benefits and barriers. This paper presents the preliminary analysis of an exploratory case study based on the interviews to relevant stakeholders of two (multinational) companies. The results show the benefits (software delivery performance) and barriers that these companies are dealing with, as well as DevOps team topology they approached during their DevOps transformation. This study aims to help practitioners and researchers to better understand DevOps transformations and the contexts where the practices worked. This, hopefully, will contribute to strengthening the evidence regarding DevOps and supporting practitioners in making better informed decisions about the return of investment when adopting DevOps.Comment: 8 pages, 1 figure, 2 tables, conferenc

Towards Guidelines for Assessing Qualities of Machine Learning Systems

Nowadays, systems containing components based on machine learning (ML) methods are becoming more widespread. In order to ensure the intended behavior of a software system, there are standards that define necessary quality aspects of the system and its components (such as ISO/IEC 25010). Due to the different nature of ML, we have to adjust quality aspects or add additional ones (such as trustworthiness) and be very precise about which aspect is really relevant for which object of interest (such as completeness of training data), and how to objectively assess adherence to quality requirements. In this article, we present the construction of a quality model (i.e., evaluation objects, quality aspects, and metrics) for an ML system based on an industrial use case. This quality model enables practitioners to specify and assess quality requirements for such kinds of ML systems objectively. In the future, we want to learn how the term quality differs between different types of ML systems and come up with general guidelines for specifying and assessing qualities of ML systems.Comment: Has been accepted at the 13th International Conference on the Quality of Information and Communications Technology QUATIC2020 (https://2020.quatic.org/). QUATIC 2020 proceedings will be included in a volume of Springer CCIS Series (Communications in Computer and Information Science

Integration of security standards in DevOps pipelines: An industry case study

In the last decade, companies adopted DevOps as a fast path to deliver software products according to customer expectations, with well aligned teams and in continuous cycles. As a basic practice, DevOps relies on pipelines that simulate factory swim-lanes. The more automation in the pipeline, the shorter a lead time is supposed to be. However, applying DevOps is challenging, particularly for industrial control systems (ICS) that support critical infrastructures and that must obey to rigorous requirements from security regulations and standards. Current research on security compliant DevOps presents open gaps for this particular domain and in general for systematic application of security standards. In this paper, we present a systematic approach to integrate standard-based security activities into DevOps pipelines and highlight their automation potential. Our intention is to share our experiences and help practitioners to overcome the trade-off between adding security activities into the development process and keeping a short lead time. We conducted an evaluation of our approach at a large industrial company considering the IEC 62443-4-1 security standard that regulates ICS. The results strengthen our confidence in the usefulness of our approach and artefacts, and in that they can support practitioners to achieve security compliance while preserving agility including short lead times.info:eu-repo/semantics/acceptedVersio

From efficiency to effectiveness: Delivering business value through software

Connected products and DevOps allow for a fundamentally different way of working in R&D. Rather than focusing on efficiency of teams, often expressed in terms of flow and number of features per sprint, we are now able to focus on the effectiveness of R&D as expressed in the amount of value created per unit of R&D. We have developed several solutions, such as HYPEX, HoliDev and hierarchical value models, but companies still experience challenges. In this paper, we provide an overview of the trends driving the transition to focusing on effectiveness, discuss the challenges that companies experience as well as the requirements for a successful transformation.\ua0\ua9 Springer Nature Switzerland AG 2019

Leveraging business transformation with machine learning experiments

The deployment of production-quality ML solutions, even for simple applications, requires significant software engineering effort. Often, companies do not fully understand the consequences and the business impact of ML-based systems, prior to the development of these systems. To minimize investment risks while evaluating the potential business impact of an ML system, companies can utilize continuous experimentation techniques. Based on action research, we report on the experience of developing and deploying a business-oriented ML-based dynamic pricing system in collaboration with a home shopping e-commerce company using a continuous experimentation (CE) approach. We identified a set of generic challenges in ML development that we present together with tactics and opportunities

Software Quality for AI : Where We Are Now?

Articial Intelligence is getting more and more popular, being adopted in a large number of applications and technology we use on a daily basis. However, a large number of Articial Intelligence applications are produced by developers without proper training on software quality practices or processes, and in general, lack in-depth knowledge regarding software engineering processes. The main reason is due to the fact that the machine-learning engineer profession has been born very recently, and currently there is a very limited number of training or guidelines on issues (such as code quality or testing) for machine learning and applications using machine learning code. In this work, we aim at highlighting the main software quality issues of Articial Intelligence systems, with a central focus on machine learning code, based on the experience of our four research groups. Moreover, we aim at dening a shared research road map, that we would like to discuss and to follow in collaboration with the workshop participants. As a result, the software quality of AI-enabled systems is often poorly tested and of very low quality.acceptedVersionPeer reviewe

From a Data Science Driven Process to a Continuous Delivery Process for Machine Learning Systems

Development of machine learning (ML) enabled applications in real-world settings is challenging and requires the consideration of sound software engineering (SE) principles and practices. A large body of knowledge exists on the use of modern approaches to developing traditional software components, but not ML components. Using exploratory case study approach, this study investigates the adoption and use of existing software development approaches, specifically continuous delivery (CD), to development of ML components. Research data was collected using a multivocal literature review (MLR) and focus group technique with ten practitioners involved in developing ML-enabled systems at a large telecommunication company. The results of our MLR show that companies do not outright apply CD to the development of ML components rather as a result of improving their development practices and infrastructure over time. A process improvement conceptual model, that includes the description of CD application to ML components is developed and initially validated in the study

Conceptualizing the Transition from Agile to DevOps: A Maturity Model for a Smarter IS Function

Part 4: IT Project ManagementInternational audienceAgile development approaches have become the norm for almost all software development now. While agile approaches can deliver more frequent releases of working software, it quickly became apparent in many organisations that they were not able to leverage these frequent releases due to the disconnect between the development and operations functions, with the latter typically responsible for releasing software to customers. This resulted in the move towards closer integration of these functions through the DevOps movement. As the trend towards digitalisation continues, companies are increasingly implementing DevOps. We propose a maturity model for this agile to DevOps transition with three levels: agile, continuous integration, continuous delivery. Based on an in-depth case study in an organisation which has several years’ experience of DevOps, we identify a fundamental disruption in the soft skills and competences that software teams are expected to possess, and in the patterns of collaboration among teams. The latter is especially salient for release managers, project managers, production engineers and even architects. Arguably, smartness may be characterized as being flexible, teaming up with people who have a different profile, belonging to a different function, and delivering more quickly what had been designed. In light of this, we argue that DevOps leads to greater smartness for the Information Systems (IS) function